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Community colleges play an important role in the U.S. economy, providing 
access to higher education for low-income young people, a path to higher-earning 
employment for low-income workers, and a supply of well-trained employees for local 
industry. In order to remain competitive with other major economies, however, the U.S. 
must sharply increase its supply of educated workers over the coming decade. 
Accordingly, policymakers and private foundations have set ambitious goals for 
improving the rate at which Americans earn college credentials. To meet these goals, 
community colleges will have to increase both the number of students they serve and the 
rate at which those students graduate. 

Improving student graduation rates will not be an easy task, particularly for 
community colleges, which disproportionately serve low-income, first-generation, and 
academically underprepared students. These students struggle with a variety of 
challenges, including job and child care responsibilities, transportation difficulties, 
financial limitations, poor high school academic preparation, and a lack of information 
about how to successfully navigate college. The goal of the CCRC Assessment of 
Evidence Series is to help community colleges identify concrete strategies that have the 
potential to improve student success on a scale needed to meet national goals for 
increased postsecondary attainment. 

Across the first year of a major grant from the Bill & Melinda Gates Foundation, 
and with supplemental funding from Lumina Foundation for Education, the Community 
College Research Center has gathered and synthesized a large body of research evidence 
regarding strategies that may improve the success of students who attend community 
college. Working papers in the CCRC Assessment of Evidence Series use the research 
literature to draw conclusions and provide concrete evidence-based recommendations to 
practitioners, policymakers, and researchers in eight major topic areas: 

• Developmental Assessment and Placement: Noting the widely shared 
view that assessment of all incoming students is crucial, Katherine Hughes 
and Judith Scott-Clayton argue that there is growing support for 
mandatory testing and placement and state-wide standardization. 

Nevertheless, some empirical evidence suggests that current assessment 



approaches do not lead to better student outcomes. The authors discuss 
emerging directions for reform, including more diagnostic and holistic 
assessments. 

• Developmental Acceleration: Ni kk i Edgecombe argues that although 
empirical evidence regarding student outcomes is limited, strategies to 
accelerate student progress through developmental education, including 
course restructuring and mainstreaming, show promise. She also discusses 
implications of the finding that most acceleration models involve 
alterations to courses or curriculum but pay little attention to pedagogical 
practices. 

• Developmental Mathematics Pedagogy: After examining six types of 
pedagogical reforms in mathematics, Michelle Hodara concludes that the 
evidence most strongly supports the effectiveness of two particular 
approaches — structured forms of student collaboration, and instruction 
that focuses on problem representation. 

• Contextualization of Basic Skills Instruction: Dolores Perin finds that 
although the evidence is only suggestive at this time, contextualization is a 
promising direction for speeding up the progress of academically 
underprepared college students. 

• Online Learning: Shanna Jaggars concludes that while online learning 
affords flexibility and convenience, students also encounter challenges in 
online coursework that contribute to low completion rates among 
community college students. She provides recommendations to improve 
online learning access and success rates. 

• Non-Academic Support: Melinda Mechur Karp argues that effective 
non-academic services achieve results through four specific mechanisms: 
creating social relationships, clarifying aspirations and commitment, 
developing college know-how, and addressing conflicting demands of 
work, family, and college. She discusses theoretical and empirical support 
for each mechanism as well as implications for college practice. 

• Institutional and Program Structure: Judith Scott-Clayton concludes 
that complex policies and structures of community colleges often confuse 
students and lead to decisions about whether and how to persist toward a 
credential that may waste time and resources and reduce the chances of 
successful outcomes. She also highlights several promising programs and 
suggests directions for future experimentation and research. 

• Organizational Improvement: Davis Jenkins argues that in order to 
increase rates of student completion on a large scale, community colleges 
will have to make fundamental changes in the way they operate. Based on 
practices found to be effective among a broad range of high-performance 
organizations, he outlines practical steps community colleges can take to 
bring about continuous improvement in student learning and progression. 
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In this introduction to the series, we describe our approach in reviewing the 
literature and discuss the primary theme of organizational redesign that weaves through 
each of the eight working papers. 



Approach to the Literature 

For each topic, we searched for relevant research literature in academic 
databases 1 and on academic and research-oriented websites 2 using a set of key search 
terms. We reviewed the abstracts of all papers matching the key search terms to 
determine whether the paper met the inclusion criteria for that topic. Inclusion criteria 
varied slightly across topics, depending on the topic scope and the quality of relevant 
literature. For example, most of the papers in the series examine research published since 
1990. For the online learning topic, however, the marked change in online technology 
and usage around the turn of the 21 st century led us to believe that research published 
prior to 2000 was likely to be dated and less relevant; accordingly, we included only 
research published since 2000. In some topic areas, the research literature was rather 
scant, and thus we included research from all educational sectors or even from outside the 
educational arena; in other topic areas, the research literature was abundant, and we were 
able to limit our scope to the postsecondary setting. Finally, some topic areas had very 
few high-quality research studies, and thus we included all relevant research, while 
putting a heavier emphasis on the findings of the high-quality studies; other topic areas 
had a larger volume of high-quality research, allowing us to discard uncontrolled or 
descriptive studies and focus on the best-quality research. 

Despite variations in inclusion criteria, each working paper takes a similar 
approach to the judgment of study quality. In general, we classified studies as: 

• Qualitative: The study did not provide a quantitative estimate of a 
particular treatment effect, but did provide other information useful for 
understanding the context of the problem and possible solutions. 

• Low Quality: The study focused only on treatment participants (for 
example, their growth over time) without comparing them to those 
who did not receive treatment. Or it attempted to compare outcomes 
between those who received treatment and those who did not, but the 
two groups were non-equivalent. For example, in many studies across 
the literatures, student participation in a given treatment was 



1 Databases included EBSCO’s Academic Search Premier, Professional Development Collection, and 
Education Research Complete; Education Full Text (Wilson); ERIC; JSTOR; Proquest Digital 
Dissertations; Social Sciences Citation Index; Google Scholar; and other databases specific to the given 
topic. For each topic, we also conducted a manual search of pertinent peer-reviewed journals, and searched 
the reference sections of previous reviews or other key influential papers. 

2 Across all topics, we searched the Institute of Education Sciences website and 19 additional websites 
belonging to community college organizations, education-oriented research centers, and education-focused 
foundations. For particular topic areas, we also searched additional research and academically oriented 
websites relevant to the topic. 
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voluntary, and no attempt was made to measure or control for pre- 
existing differences between volunteers and non- volunteers. 

• Better Quality: The study compared outcomes between those who 
received treatment and those who did not. While groups were neither 
randomized nor perfectly comparable at the outset, the authors 
attempted to statistically control for observable differences between 
the groups (using, for example, regressions controlling for multiple 
covariates). Or the study relied on some known source of variation in 
treatment status that is plausibly unrelated to individual participation 
decisions (for example, regression continuity designs). Randomized 
studies with obvious flaws (for example, failing to control for 
differential attrition) also generally fell into this category. 

• Randomized: The study explicitly randomized students into treatment 
and control groups, allowing a clean and clear measurement of the 
treatment effect. It is important to note that randomized studies are 
often conducted with small and sometimes unrepresentative sets of 
students. Accordingly, their results cannot always be generalized to a 
larger population of interest. 

We used our judgment of study quality to assess the nature and strength of the 
research evidence within each topic area. Based on that assessment, we reached 
conclusions and recommendations using a variety of sources, including qualitative and 
survey research, the theoretical literature, and practitioner input. Further input from 
experts in each field as well as from our research advisory panel of community college 
leaders and researchers 3 helped us refine and extend recommendations in an attempt to 
make them concrete and practical. 



3 We extend many thanks and appreciation to our Bill & Melinda Gates Foundation grant advisory board: 
Thomas Brock, Director, Young Adults and Postsecondary Education Policy, MDRC; Anthony Bryk, 
President, The Carnegie Foundation for the Advancement of Teaching; W. Norton Grubb, David Pierpont 
Gardner Professor in Fligher Education, University of California at Berkeley Graduate School of 
Education; James Jacobs, President, Macomb Community College; Robert Johnstone, Dean of Planning, 
Research, and Institutional Effectiveness, Skyline College; Henry Levin, William H. Kilpatrick Professor 
of Economics and Education, Teachers College, Columbia University; Bridget Terry Long, Professor of 
Education and Economics, Harvard Graduate School of Education; Kay McClenney, Director, Center for 
Community College Student Engagement; Donna McKusick, Dean of Developmental Education and 
Special Academic Programs, Community College of Baltimore County; Scott Ralls, President, North 
Carolina Community College System; Daniel Solorzano, Professor of Social Science and Comparative 
Education and Director of UC/ ACCORD, University of California, Los Angeles; Karen Stout, President, 
Montgomery County Community College; Susan Wood, Vice Chancellor, Academic Services and 
Research, Virginia Community College System; and Holly Zanville, Senior Program Director, Lumina 
Foundation for Education. 
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Major Theme: Organizational Redesign 



In his paper on organizational improvement and college redesign, Jenkins argues 
that in order to improve, colleges need to move beyond implementation of small-scale 
programs by cultivating a college culture that encourages and sustains wide-scale 
institutional reform. In our initial conceptualization of this series, we approached this area 
as one important stand-alone topic. Over time, however, we came to realize that 
organizational redesign is a theme that is integral to all eight papers. To substantially 
improve developmental education, online and face-to-face pedagogy, or academic and 
non-academic support services, the entire institution must be involved with and 
committed to fundamental change and improvement. Moreover, no single strategy in 
isolation will increase student success rates on a substantial scale; rather, strategies must 
work together in concert across an institution. Keeping these principles in mind, the 
papers in the series highlight specific ways that college processes and services can be re- 
designed to improve student success. Overall, the papers in this series build the 
foundation for four broad recommendations: 

(I) Colleges should work to simplify the structures and bureaucracies that students 
must navigate. 

To the incoming community college student, college represents a confusing and 
complex tangle of restrictions, recommendations, and requirements. Some rules and 
practices are in place because they support student success, but others are in place due to 
tradition, convenience, or inertia. For example: 

• Hughes and Scott-Clayton’s working paper on developmental education 
placement testing notes that almost all community colleges use 
commercially developed tests as the primary metric for placement of 
students into developmental education, despite the fact that these exams 
do not support colleges’ efforts to provide appropriate programming and 
services to academically underprepared students. While they are 
convenient mechanisms for screening, these tests represent a barrier to 
student progression, and they provide little diagnostic information for 
students or faculty members concerning the particular problems individual 
students face and must overcome. 

• In her working paper on institutional and program structure, Scott-Clayton 
cites a variety of common policies that may confuse students or 
inadvertently hinder their progression. For example, in some community 
colleges, a portion of noncredit programming is quite similar to credit 
programs in curriculum and quality, and may cost as much or even more 
per term, but if a noncredit student decides to switch to an academic 
program, the courses he or she has already taken are not applicable toward 
an academic credential. 

• Registration and financial aid processes can be extraordinarily and 
unnecessarily frustrating. Anecdotally, a president of one college recently 
discovered that many of his students waited at the financial aid office for 
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over an hour to reach the front of the line, only to be told they needed to 
return after filling out a necessary form. (The college now has a staff 
member meet students as they walk in the door to ensure they have the 
proper forms before entering the line.) 

• Most community colleges offer an impressive array of academic 

programs, but many do not clearly map out their offerings in a way that 
makes it clear to students, particularly those who lack clear goals for 
college and careers, what the employment and further education goals 
particular programs are designed to lead to, and how students can 
successfully navigate program requirements to complete as quickly as 
possible. 

Such policies and practices create unnecessary obstacles to student success. For 
how many new students, many of whom are balancing complex work and family lives, 
might these bureaucratic barriers constitute the “last straw” that encourages them to walk 
away from college entirely? Colleges need to re-examine policies, practices, and services, 
to ensure that they are all aligned with the goal of student success. To do this, Jenkins 
recommends that colleges form cross-functional committees or task forces of faculty, 
student services staff, and administrators to map out the experience of students from the 
time they first make contact with the college, examine the interactions between students 
and college programs and services at each point along this “pathway,” and assess the 
extent to which college policies and practices help or hinder students from making 
progress toward successful completion. 

(2) Broad engagement of all faculty should become the foundation for policies and 
practices to increase student success. This should include active faculty involvement in 
student support activities. 

Jenkins highlights that a variety of research on effective organizations across 
multiple fields converges in a clear consensus that substantial organizational 
improvement is unlikely to occur without strong employee involvement. Considering 
these findings in light of the decentralized nature of authority in community colleges, it is 
clear that colleges cannot enact systematic reforms from the top down; faculty and staff 
must be motivated participants in efforts to improve organizational effectiveness. 

Several papers in this series join in the conclusion that student support structures 
should be integrated into students’ daily academic experience, which would require 
active faculty involvement in support activities. One clear explanation for this is that, for 
many reasons, students who need supports may never seek them out. Students may not 
think they need help; they may not know the services exist; they may think college- 
provided services will not be useful; they may feel intimidated about approaching others 
for help; they may be confused about how to find or use the service; or they may feel that 
using the support would flag them as being unworthy, unintelligent, or “not college 
material.” 
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In her paper on online learning, Jaggars points out that many student services are 
offered only on the physical campus; she argues that online students are much more 
likely to use these services if the supports are integrated into activities in the online 
course itself. Similarly, in her paper on what makes non-academic supports effective, 
Karp recommends that colleges include non-academic support activities into the overall 
curriculum through student success courses and activities in academic courses. She notes, 
for example, that “math faculty might find ways to use the FAFSA in their courses to 
help students learn math skills while also being exposed to the financial aid process. By 
integrating non-academic supports into the ‘regular’ curriculum, students will not need to 
seek out such supports and are more likely to encounter them on a regular basis.” To 
incorporate such activities into the curriculum, faculty support is critical. 

It is important to recognize, as Jenkins points out, that previous attempts to 
engage faculty in improvement efforts have not always fared well. Perhaps these efforts 
have been stymied by a focus on student retention and completion, which are measures of 
institutional effectiveness, rather than by a focus on student learning, which is a measure 
of instructional effectiveness. To engage faculty, colleges may need to empower them to 
establish common learning outcomes and assessments for academic programs, as 
discussed further below. 

(3) Colleges should be encouraged to align course curricula , define common learning 
outcomes and assessments, and set high standards for those outcomes. 

Based on his review of research on K-12 school improvement, Jenkins finds that 
schools that are effective in serving educationally disadvantaged students are 
characterized by “instructional program coherence,” meaning that courses and teaching 
are guided by a common instructional framework, with clearly defined learning outcomes 
and integrated assessment and academic supports. Community college programs are often 
weak in this regard. For example, developmental education courses are meant to prepare 
students for college-level math and English courses; they thereby prepare students to 
succeed in college-level work more generally. However, the material covered in 
developmental courses is often misaligned with the skills necessary to succeed in college- 
level courses (a topic of ongoing CCRC research). In her paper on developmental 
education acceleration, Edgecombe notes that many acceleration programs have taken 
aim at this problem — they are designed to improve student success by eliminating 
redundant or misaligned material within a given program’s curriculum. 

Faculty within individual departments need to work together to make the goals of 
instruction explicit, providing a basis on which to measure actual learning outcomes and 
assess the quality of instruction. Popular performance metrics, such as course grades or 
student evaluations, tend to be inconsistent within and across instructors over time. 
Learning outcomes, on the other hand, are defined by faculty involved in course design 
and instruction, are measured consistently across all sections of a course (and can often 
double as the course’s common final exam or project), and can build across courses 
toward overall program learning outcomes. As Perin points out in her working paper on 
contextualization of basic skills, the process of defining course learning outcomes can 
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also help faculty from different disciplines communicate and align expectations across 
courses in terms of key reading, writing, and math s ki lls. Moreover, learning outcomes 
can serve as the basis for a peer-review-driven continuous quality improvement process 
within individual departments and across the institution (see below). In her paper on 
online learning, Jaggars argues that now is an excellent time to introduce such a process 
into online learning programs. Given the relatively new environment of online learning, 
online programs might afford colleges an opportunity to implement continuous 
improvement in a context that may be more open to innovation than is the relatively 
traditional environment of face-to-face learning. 

(4) Colleges should collect and use data to inform a continuous improvement process. 

Using measurement and evidence to inform management decisions is a central 
feature of effective organizational improvement models outside of education. It is 
likewise a key part of K-12 school reform efforts and a primary tenet of college 
improvement models such as Achieving the Dream and the Academic Quality 
Improvement Program. Such models emphasize that major improvements to a system can 
best be achieved through a process of examining key outcomes, enacting policies that 
attempt to improve those outcomes, and re-examining outcomes, in a continuous cycle. 

Yet, as Jenkins notes in his paper, a national survey of community college 
institutional research practices found that top administrators generally do not use data on 
student outcomes for decision making. In fact, community colleges often allocate limited 
resources for institutional research; many treat it as an administrative function that is not 
central to the management of the college. Perhaps this should not surprise. Given that 
college funding is based on enrollments rather than outcomes, there are few incentives for 
college leaders to focus on the latter. States can motivate colleges to move toward data- 
informed decision making by tracking student attainment of intermediate and completion 
milestones, engaging colleges throughout the state to examine the data, supporting efforts 
by colleges to use their own data to improve programs and services, and creating policy 
incentives for systemic reforms. 

Jenkins’s paper also outlines a five-step continuous improvement process 
(employing measures of student learning and progression) that colleges can follow as part 
of an overall effort to redesign themselves for improved student completion. Each of the 
other papers in this series includes recommendations for specific actions colleges can 
take to improve student success at each stage of students’ experience in college. 



As noted below, Jenkins’ paper on organizational improvement, Scott-Clayton’s 
paper on student navigation of institutional and program structure, and Jaggars’ 
paper on online learning are now available. Other working papers will appear in the 
coming weeks. All papers may be downloaded from the CCRC website 
(http://ccrc.tc.columbia.edu) as they are released. 
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Edgecombe, N. (2011, forthcoming). Accelerating the academic achievement of students 
referred to developmental education: A review of the evidence (CCRC Working 
Paper, Assessment of Evidence Series). Anticipated release: March 2011. 

Hodara, M. (2011, forthcoming). Reforming mathematics classroom pedagogy: 

Evidence-based findings and recommendations for the developmental math 
classroom (CCRC Working Paper, Assessment of Evidence Series). Anticipated 
release: February 2011. 

Hughes, K. & Scott-Clayton, J. (2011, forthcoming). Assessing developmental 

assessment in community colleges (CCRC Working Paper, Assessment of 
Evidence Series). Anticipated release: February 2011. 

Jaggars, S. S. (2011). Online learning: Does it help low-income and underprepared 
students? (CCRC Working Paper No. 26, Assessment of Evidence Series). A 
Brief of this paper will also be made available. 

Jenkins, D. (2011). Redesigning community colleges for completion: Lessons from 
research on high-performance organizations (CCRC Working Paper No. 24, 
Assessment of Evidence Series). A Brief of this paper is also available. 

Karp, M. M. (2011, forthcoming). Toward a new understanding of non- academic student 
support: Four mechanisms encouraging positive student outcomes in the 
community college (CCRC Working Paper, Assessment of Evidence Series). 
Anticipated release: February 2011. 

Perin, D. (2011, forthcoming) Facilitating student achievement through contextualization 
(CCRC Working Paper, Assessment of Evidence Series). Anticipated release: 
February 2011. 

Scott-Clayton, J. (2011). The shapeless river: Does a lack of structure inhibit students’ 
progress at community colleges? (CCRC Working Paper No. 25, Assessment of 
Evidence Series). A Brief of this paper is also available. 
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